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1*1 fbe ala of artificial Intelligence research is to 
produce human-like behaviour outside a living organics* It is 
activated by a desire to understand t he mechanlmi of thinking 
and learning in human beings* there exists no general state* 
sent of the problem of artificial intelligence# fie search 
efforts in this field have been mainly directed towards solu- 
tion of specific problems like game playing# theorem proving# 
pattern recognition and simulation of cognitive processes* k 
human being perfuming these tasks would be called • intelligent^: 
any artifact which performed the same tasks# either in the same 
fashion as humans do or Incorporating different information 
processing techniques mould be called •artificially intelligent** 
these artifact may be mechanical or electromechanical systems 
{examples of which will be mentioned later on) # hardware comp* 
rising of electronic components# or as is largely the ease today# 
digital computers programmed to perform these ta sks * Computer 
simulation of human behaviour is popular for the following 
reasons* 

a) its high speed computational and symbol 

-manipulation abilities! 

** 

b) its vast •mamorf* # there information earn be 
stored and retrieved when needed# nod# 

e) Its versatility*® gamers! purpose computer earn 
be made to perform a variety of specific tasks 
by simply providing it with a suitable programs* 

4 iso* ft imiigF tycNB ^MWftft tte 


hypotheses about the proves©* underlying 
the behaviour# i® such, it can be quickly 
tasted by fomlsg the program on the computer 
and modified# If necessary* This process can Us 
repeated till a satisfactory solution is obtained* 

1.2 Beraons trying to simulate various aspects of toman 
behaviour on a computer using the tec~mlque mentioned above fdt* 
one group of artificial intelligence researchers* there is a 
second group which easts to find new and tetter uses of computers 
Uy expanding the intellectual capacity of the machine 9 using 
whatever means are applicable* they are interested In creating 
computer programmes which manifest the kinds of behaviour m want* 
though It may not adopt human-like processes and techniques* 
there is a third group which believes that research in artificial 
intelligence can lead to better uses of machines by men than 
are being made today* Instead of the natural division of labor 
between a man and the machine * most routine work dene by the 
machine and all generalisation and higher-level t h i n k i n g by man * 
they believe in mere cooperation and adaptive participation from 
machines In attacking implex and difficult problems* Their 
emphasis Is on close interaction batsmen man told machines* 

1*8 in early attempt to model some aspect of h uman be* 
haviour was made by does is toy Ushby, i960)* m his beck 
•Design far a Stain* he describes the design end construction 
of an electromechanical device called the homeostat* Thin devise 
models the adaptive behaviour of the human nervous system* 


Grey Walter* e *M*Spec!3iatrix* la a model of elementary reflex 
benavloctr found In hcmans (Walter 9 1961) • 

1*4 Probably the first serious attempt to simulate 
luman theorem proving behaviour on a computer is a computer 
programme called the Logic Theorist developed in 1986 by Newell, 
Shaw and Simon on the E4KO JQHNIic computer* ilthough the 
prograame could not model a brilliant logician, It made a signi- 
ficant landmark, In the sense that it helped psychologists and 
computer scientists to gain deeper insight into problems and 
techniques related to understanding human behaviour. It also 
started a atoole new trend of symbol manipulation and so-called 
heuristic programming* Briefly, a heuristic programme is one 
ihich incorporates a role of thumb, strategy, trick or any 
other kind of device *hich drastically limits search for 
solution to large complicated problems# Heuristics do not 
guarantee a solution to a problem; In this regard they ate 
different from algorithms vtoieh always wUi lead to a solution* 
But Where algorithms are Impractical as in the csss of complex 
problems, like chess-playing f or exmaple, heuristic methods 
offsr more general applicability* The logic Theorist was used 
to discover proofs to theorems In syaabolio logic* It is 
interesting to note that the Logic theorist gw rise to the 
first lfet-prosesslng computer language, called the Information 
Processing Language or XH*« 

The work of Geismte* (Geiernter, 1968) extends 
heuristic programming Lises to pro eft of theorems in Bud losau 



Felgenbaum 1 s SPAM program, or the momentary 
Perce Ivor and Memorlzer, models verbal learning behaviour* 

It simulates the information-processing activity underlying 
human association learning* 

Howell* s Chess-machine and Samuel* s Checker playing 
programme are examples of the fact that programmes can he 
written to exhibit * learning* behaviour, though Internally 
they contain no hasan-llke problem-solving and learning processes* 
The programmes are able to Improve their performance as they 
gain play experience * Samuel* s Checker Programme makes use 
of a linear polynomial expression with Ingenious ways of 
selecting and chan&glng Its terms and of determining and 
modifying corresponding coefficients* The terms represent 
characteristic s of game situation such as mobility, advancement 
and centre control* 

Besides therem-proving and game playing, attempts 
at the solution of real problems have also being made* 1 
good example Is Tonga* s Issembly-llne balancing programme 
(Tenge* 196s) • This Is an example of application of heuristic 
programming to an important management science problem* 

Balancing and assembly-line Involves finding and efficient 
arrangement of workers, tasks and work stations so as to 
maximise the rate of assembly or minimise the number of 
workers needed* 

The above efforts represent attempts' at solutions 
to specific problems only* These programmes cannot be used 
under varying ©Ireuastaneee* for example, the checker - playing 
programs cannot be made to play chess or even tlctactoe* 
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aspects of intellectual behaviour is that investigation 
and evaluation or made easier* ifter enough information 
about different aspects of behaviour Is accumulated ,some 
common features may he abstracted which could he used la more 
general problem-solving situations* 41 so. It Is preferable 
to concentrate on particular problems If artificial intelligence 
research is to be of use in solving some Immediate problems* 

There have been attempts toward construction of 
more general intelligent systems* The General Problem Solver 
or GPS developed by Newell, Shaw and Simon is the first 
large-scale attempt of this kind* They used their earlier 
experience with the Logic Theorist in the design of GPS* some 
of the common features observed to be generally useful 
Xheuri sties have been abstracted, and these form the * general- 
part* of GPS, separate from the problem-specific part of the 

programme* GPS can handle a variety of problems like solving ; 

| 

r 

trigonometric Identities, compile computer programmes and to 

1 

prove theorems in the propositional calculus* The most 
extensively used technique In GPS is the * means-ends analysis* • 
in initial problem state Is transformed into the desired target j 

f 

state by selecting and applying operations which, step by step, j 
reduce the difference between the two states* J 

Hermann* s computer programme called *6ate* Is another ; 
attempt toward a sore general system* In Galea, the learning 
mechanisms fora the Important parts Gate can find solutions 
to progressively more difficult ’T©wer-of#Sanoi* problem# 
using its previous experience* Gate and GPS will be treated 
in come detail In a subsequent chapter* 



1*5 Some of tiie current difficulties faced in this 
field of research lie in trying to find ont how computers 
can leam new heuristic aetods and rules and how the 
induction capability found in hraaans cun he Mechanised. 

The most important problem is in overcoming the man-machine 
communication barrier. is Simon has indicated y the dilemma 
is that we can design acre intelligent machines if we could 
communicate with them better but we could communicate with 


them better if they were more intelligent* 

1*6 In this thesis we will be mainly concerned with the 


aspect of * learning* i£ artificial systems* The-nest-chapter 
wiilJbe- .concerned with theories of Learning and some of the 


programmes for machine learning will be 






1 Learning programme which ’learns* to play the game of 
\ tictactoe wpl be presented 



CHOTBR TWO 


theories of mmsim 

2*1 In this chapter we shall briefly deal with the theory 

of learning and see how it is possible to exhibit intelligent 
behaviour in artificial systems, particularly in computers* 

Several definitions of the term "learning 1 ' have been proposed, 
each suitable for the particular physiological process lx* .that 
is being described or modeled* The learning process involves 
modification of present actions through past experience* But 
mere modification is not enough* for example, a seagull which 
follows ships in search of food is said to have "learned" how 
to find food* If, on seeing a ship, it always flew in the opposite 
direction, it would have had its behaviour modified, but would 
not be said to have "learned" • (Humphrey, 1983)* Therefore, 
behaviour modification that is useful to the organism can be 
called learning* Humphrey proposes a continuum of learning, 
starting from simple adaptation or accommodation at one end, 
and m&se~learnlng, puszle-box learning and ape*learning, in 
stages of increasing complexity, leading to human learning at 
the other end* Pavlov* s conditioned response can also be placed 
in this continuum* 

Conditioned response implies associative learning 
as displayed by Pavlov* s dog* Salivation at the aitBxof sight 
of food is the "unconditioned stimulus", and then, when the 
food is presented and a bill rings at the same time, the bell 
may be called "conditioned stimulus** Conditioning occurs when 
the response of salivation is excited by the bell alone (now 
failed the conditioned response) without the presence of fbod* 



Post of the modern dlfinitlons of learning be*© been 
In terms of conditioning and the problem of learning Is reduced 
to the problem of "reinforcement" • Reinforcement has been 
defined thus • 

*4ny stimulus which can increase the strength of a 
response when presented in close temporal conjunction 
with the occurrence of the response* (Deese, 1952) * 
Three definitions of learning will now be looted* 
Guthrie's definition (Guthrie, 1955) reads* 

* Learning is the ability to respond differently 
to a situation because of past responses to the 
situation* • 

Hilgard* s definition (Hilgard , 1948) reads* 

* Learning is the process by which an act iy it 31 
originates or Is changed through reinforcement 
procedures as distinguished from changes by 
factors not attributable to reinforcement* • 

Thorpe's definition (Thorpe, 1950) reads* 

* The process which produces adaptive change in 

! 

individual behaviour as the result of experience** 
Cannon (C 3 nnon, 1929) takes the view that learning is inevitable \ 
for survival and calls it the process by which the behaviour 
is, in some way, modified by external or internal changes 
towards the survival of the organism* T^ls is, of course, only | 
slightly different fro® Humphrey' s viewpoint# 

separate definitions have been proposed for .several 
types of learning like, *trial-and*error learning*, "latent 
learning*, and * insight learning** Learning may ooour when 
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trlal-and-error learning. Some organisms can leam without 
Immediately manifesting their learning In a changed performance* 
This Is latent - learning* 

2*2 The effect of learning In a human being is an 

increased ability to adapt himself to new situations* The 
process of adaptation may be looked upon as a specific 
problem-solv ing process or directing to seek a particular 
goal* Given a particular problem-solving situation, a person 
utilizes his earlier experience in solving similar or related 
problems and gradually arrives at the correct solution. He 
may display occasional flashes of insight, thus speeding up 
the problem-solving process* How can this process be 
mechanised? Hr more particularly, how can a computer be 
programmed to sltelate this problem-solving behaviour? 

2*5 k computer possesses some of the requirements that 
are necessary to achieve the above end. It offers a large 
memory- space where information can be stored, input - output 
facilities for communication with the programmer and hardware 
for actual information processing - symbol manipulation and 
computation* It onhy remains to tell the computer how to 
attempt trial solutions for a problem, store the results of 
these efforts and utilize them in solving new problems* This 
can be done by providing It with a programme incorporating 
these features* The programme may also tell the computer 
how to generalise probiemosoivlng situations so that it is 
able to solve as large a class of problems as possible* 
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1 computer capable of such generalisation can Be said to 
possess the ability of indactise inference* This utilisation 
of past experience is a form of feedback of information. 
Programmes utilizing this form of information feedback will 
Be described in the next chapter* 
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mkvmt, THU® mommam comfsms to issbk 

5*1 In this chapter, m shall, briefly consider some of 

the computer programmes for learning that hare been proposed* 
these programmes Illustrate how computers can learn, utilising 
their previous experience In similar situations* We shall 
consider the following systems $ Hermann’s *Gate* - the . 
artificial student, Newell » Shaw and Simon* s General Problem 
Solver, Oettlnger* s learning Programme, shannon’s Chess Machine 
and Samuel’s Checker Player* 

5.2. Hormarm’s (1962, 1964) learning system incorporates 
some higher-order feedback features* feedback allows a system 
to interact with the environment and correct its behaviour by 
coming nea rer to Its goal - as In a servomechanism* But there 
is no facility for recording and using Its past experience! 

If the task were to be repeated, the same trial-and-error 
behaviour would be repeated before proper adjustment was made* 
Previous experience can be conveniently stored In the computer 
and utilised in order to avoid repeating the same mistakes* 

Gate Is the Japanese word for learning* It consists 
of hierarchically interacting feedback-loop units, each of 
which is equipped with general rules for deo ision-aaklng 
and information-handling within its prescribed domain of 
authority and activities* 

It has four mechanisms, which are em bedded in a 
master feedback unit called the' mechanics coordinator* this 
mechanism coordinator, as well as each mechanism is designed 



to apply a Paste cycling process involving three phases 
(see figure 3.1)* 4n analysis and t»st phase, a tentative 
selection or correction phasef, and a consequence generation 
phase* 4 feedback loop is formed when the analysis and test 
phase receive? the output of the consequence generation phase* 
Upon re-entering the analysis and test phase, reformulation 
of the given task is done by comparing the consequences with 
the description of the task* 4 fresh selection or a correction 
of the previous one is effected in the tentative selection or 
correction phase* The three phases are repeated until a success 
or failure is determined by the analysis and test phase* 

This general structure is common to all four 
mechanisms and the mechanism coordinator* The four mechanisms 
have the different functions of planning, programming, problem- 
solving, and induction* The programming mechanism generates 
internal programmes from externally given description of the 
task* The planning mechanism divides the given task into 
sub-tasks which are easier to perform* These sub-tasks are 
solved by the problem-oriented mechanism* Efficient use of 
past experience is made using the induction mechanism which 
surveys the system* s previous experience with similar tasks 
and applies it to new situations * Since humans ere better than 
machines in planning and induction, in a partnership between 
man and this system, man can, through the mechanism coordinator 
give suggestion about new conjectures to be tried or subgoals 
to be set* 

Qaka* s performance has been tested using a sequence 
of problems from the •Tower-of-Hanoi* puszle* Gate was able 
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to find solutions to progressively more difficult Tower-of- 
Hanoi problems, each time utilizing Its past experience and 
finally finding a general solution to xfeb® pattern through Its 
Induction mechanism . 

3*5 Learning possibilities In Newell, Sham and Simon* s 

(Newell, 1960) General Problem Solver or GPS have been analysed 
by them, is already mentioned, GPS is a programme that 
incorporates heuristic means for solving a substantial range 
of problems, for example, discovering proofs for theorems in 
logic, proving algebraic and trigonometric identities and 
performing formal integration and differentiation, 4 brief 
description of GPS follows s 

GPS is a programme for working on tasks in an 
environment const feting of "objects* and "operators** Symbolic 
logic is one particular such task environment, in which GPS 
can operate * Here, the objects are logic expressions; the 
operators are the allowable rules of logic for transforming 
one expression to another • By successively operating on a 
given logical expression, it is simplified to the required 
form* But the object and operator are not confined to logic* 

GPS generally operates this way# it detects "differences" 
between objects and organises the information about the task 
environment to goals* There are three types of goals: ‘ 

Transform object 4 to B, ! 

Reduce difference £> between object 4 and B, 

4pply operator R to object 4* 

The goals are achieved by setting up suVgoals, whose attainment j 
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leads t,o the next sob goal to be achieved* The baste organisation 
of GPS for this purpose Is shown In figure 3.2. It evaluates 
the goal to see whether It should be worked on* If It accepts 
the goal, Ids it selects a method associated with the goal* 

If the method falls, It selects another method and applies It 
to achieve the goal* Figure 3*5 sho^e the methods employed 
for achieving the three goals* Thus, to transform and object 
4 into an object 3, the objects are first compared element by 
element. If a difference is revealed, then a subgoal is set 
up to reduce the difference* If this subgoal Is attained , 
a new object will be produced which no longer has the same 
difference noted earlier • k new subgoal is now created which 
will further reduce the difference • This process is repeated 
until no difference is detected* The entire goal will then 
have been achieved* 

Learning In GPS is achieved by having the learning 
situation shown In figure 3*4* k learning situation requires 
another programme called the learning Programme* The performance 
programme at the bottom of the figure Is GPS* The learning 
programme operates on the performance programme to produce 
a new performance programme better adapted to Its task* GPS 
will thus become a better problem solver* The performance 
programme has tee parts i Cl) GPS proper, with its goal types 
and methods which are Independent of the subject matter; and j 
(2) the specification of the particular task environment: Its j 
objects, operators and its differences* 4 fuller description | 


I 
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of tlie GPS and analysis of the learning situation can be 
found In the publication cited above* 

5.4 Qettinger (1952) describes some technique* by which 

a computer can be made to display learning. He dlytded the 
computer Into two parts, letting one part play the role of 
the learning machine and the other, the erwlorzment* Be 
Interpreted the learning activities teas a series of shopping 
expeditions , where the shops are described by an m a m matt lxl 
the elements ctcj # 4 if shop i- has article j , 

oth.«4» a r o. to., *><» «MM .hop MM *Bd 
columns, article vectors* 4fter an initial period of trial- 
and-error, the computer would know which shop contained a 
specific article and also what articles any given shop stocked* 
This displayed an elementary ease of learning* 

5*5 Most of the current day efforts however, have been 

In programming computer© to learn to play games, particularly 
chess and checkers* The earliest scheme ms proposed toy 
Shannon (Shannon, 1950)* He divided the problem Into two parts $ 
(1) (Shoeing a suitable code to represent the positions and 
pieces on a chess board as numbers} (2) chosing a strategy 
of play* The second part ms the most difficult part of the 
problem, since the straightforward strategy of calculating 
all possible variations of all moves to the end of a game is 
beyond the capabilities of even present-day computers* In a 
typical chess position there will be about 52 possible moves 
with 52 possible replies— already this creates 1024 poasibiiitia 

f 

Host chess games last for about forty moves or more foe each ! 







side* The total umber of possible variations in an average 
game is about i0 120. k machine calculating one move each 
millionth of a second would retire many* mam years to make 
its first move! So, Shannon used only few good strategies > 
his computer could play only a reasonably skillful game* 

Shannon* s programme did not possess any learning 
abilities* Newell {Newell, 1965b) has proposed a programme 
which incorporates learning* His machine works with a set of 
expressions or rules of thumb thich provide it with the ability 
to play a reasonably good game* Following a sequence of games, 
it generates new expressions and modifies previous ones, thus 
evolving a set of expressions or rules of sufficient complexity 
to play good chess - 

Many other chess programmes have been suggested and 
a specification and comparison of them could be found in 
Newell, Shaw and Simon (1963) * 

3*6 Samuel (1963) made extensive studies in machine 

learning using the game of checkers* He preferred checkers 
to chess because the simplicity of its fules permit greater 
emphasis to be placed on learning techniques. His ocmputerx 
plays by looking ahead a few moves and by evaluating the 
resulting board positions, much like a human player. The 
board positions are • scored* in terms of its value to the 
machine with respect to advancement, mobility and centre control* ! 
Storing is done in terms of a linear polynomial, its terms j 

and coefficients represent evaluation of the properties of | 

Various board positions mentioned above- Besides these, the j 
parameters considered are relative piece advantage, number of 



kings and their 'fain® in terms of pieces, and tilt inability 
for one sloe or the other to- awe which would decide the end 
of the game • Using these techniques, his machine learned to 
play a rery good game of checkers. 

8*7 In this chapter m hare briefly considered several 

programmes for computers to learn* In the next chapter a 
specific learning scheme for a computer to learn to play the 
gam# of tictaotde will he presented# 
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4*1 In this chapter, a ccmpater programs ■which learns 

to play the game of tictactoe using a scheme of learning mill 
he presented* The computer mill gradually Improve Its ability 
to defend itself against the opponent* learning mill he said 
to he complete when ccmpater no longer loses a game* 

4*2 ‘She Gsme*- 

Tictactoe or, nought-and-erosses as It Is also known, 
Is a simple game played between two opponents* The tictactoe 
•hoard* or simply, "the hoard* as it will henceforth he referred 
to, consists of nine squares and the two players alternately 
marl: a square with a nought (0) and a cross (X)* The first 
player to hare a row or a cdlnmn or any of the two diagonals 
filled with his symbol (either a nought or a cross) Is the 
winner* Game theory would describe this game as a two-person, 
aero-sum game* And, both players can adopt strategies which | 
will always lead to a draw* Thus, our criterion of successful : 
learning by the computer will he its ability to draw the game 

i. 

against any player* j 

4*8 Modus Operandls- ; 

(s) The first thing is to amber the squares 
of the board, using the numbers one to nine inclusive* The 
choice of ambers Is Important because It directly affects 
the statement of a winning position* There are cfevlcu&y 
many ways of carrying .out this amber Ing, but this terms out 
to be a matter of no Importance since the computer, provided 
it is hold which game is wen and which lost, will still be 



able to learn to play the game, regardless of the umbering 
system need* The umbering ue»d In the present ease Is shown 
In figure 4*1* The reason for choosing this scheme was that 
we plan to mate the first two mores of every game compulsory* 
This was necessary in order to decrease the demand on the 
computer memory space* 

(b) Baring decided upon thp kind of umbering to 
be used, we mast next provide the computer the ability to 
discover whether a game Is won, lost or drawn* For this 
purpose, we have the following arrangements 

Each time the opponent selects a square, bhgt 
square will be given a weight 1* The square selected by the 
learning programme will be given a weight 4* It Is clearly 
seen that if the sum of any row or colimn or diagonal squares 
equals 3, the learning programme has lost; If any sum is 12, 
the learning programme has won* if neither, and all squares 
are full, the game is a draw* 

(c) Assuming that the computer is given instructions 
as to how to select the squares, it must now learn that certain ; 
ways of carrying this process of selection lead to a win, others 
to a loss* In order that It will be possible to decide whether j 
a move Is good or bad, a complete record of the set of moves j 

i 

making up a' game Should be kept* There are nine or less moves j 
in every complete game; the computer should be able to store i 

t 

all sequences (of different •lengths*) arising from all possible 

i 

combination of the numbers 1 through f « There are more than 
IW* &m «ich sequences; the computer does act have such a large 
number of storage locations* Therefore, to reduce the amber 


of storage locations retained the first two mores of every 
game are fixed to start with* Thus, the centre square and the 
top left hand square (Squares numbered 8 and 9 In figure 4*1} 
will always he the first too mores of every game* 
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figure 4*1 

The rest of the squares are umbered 1 through 7* There are 

18,700 sequences possible* 411 sequences are Initially given 

a "tactical value" which Is positive* If a particular sequence 

Is discovered to have led to a loss, its tactical value will he 

Immediately mace negative ; the computer will not make choices 

give 

which wlii^rise to aieh sequences* Before malting a choice, 
the computer will select a square and check the tactical value 
of the sequence formed hy the numbers selected so far* If the 
tactical Value Is positive, It will "announce* that selection 
as Its choice I If It Is negative, it will make a fresh selection 
and check the tactical value of the new sequence* If the 
tactical value is again negative it will make yet another 
selection* Thus the computer will finally select a number 
which gives rise to a sequence with s positive tactical value* 



a set of .981000068 which load to a loss and these will b are 
a npgatlwe tactical -vale® associated with each* Ml the other 
seioeoces 8111 tote positte tactical ral a# associated wltii them 
and the combination of nosaers* Siting rise to these sequences 
will be *safe* choices for the computer against that player* 
isauadng a regular pattern of play by the cppon emt, w© hate 
the following Illustrations of games (the first mote Is always 
by the opponent) , l standing for lost and D for drawn! 

8-9«*-S-5-l 

S*9*l*4**IIWk 

8»9«i«*5»2»3»6»Ii 

8*9«i*5*2^»S*7*4*B 

ifter this point the game remains constant If the opponent ; 

plays consistently* we may also note that the first star : 

sequences now bays a negative tactical tame associated with 
each of them* 

Exactly the same process occurs with different 
opponents, although it tabes much longer to reach a stable ! 
state* the computer can easily perform the operations described, 

and It tery quickly learns the Important® tactical steps, j 

( 

after which it never loses* i^other Important feature is I 

tint the computer does not hate the same statement of tactics j 


* It should be noted that the names •digit* t "Mbaf** •»®ssre% 

•selection?* ,* choice* cm all us ed , inter changeab l y i they ell 




aa t he opponent* T^e opponent plays dednctitely whereas the 
computer carries through the steps of checking the tactical 
talues treat the store and finding cot another earlier experience 
in a stellar situation has been against it# 

(d) The method of storage Is considered next# Bach of 
the 15,700 possible sequences will hate its associated tactical 
tain® stored at a particular location# To find the address 
shore the tactical talus of a particular sequence which has 
appeared im a game Is available is a problem In itself# for 
this purpose} a formula Is deteloped which will calculate the 
correct address using a umber asking up the sequence and. the 
order in which they appeared* ee first hat© the' following 
arrangement: 

1# 4H sequences with only one digit hate their 
tactical t a lues stored In locations 1 through 7* (Since the 
first two motes are compulsory} our sequences will maSm up of 
numbers 1 through 7 only) • 

2* There are 41 sequences hating oi&y two digits# 

Tnese hate their tactical talues stored in locations S through : 
47* ind so on* Table £ gites the corresponding areas in the ! 

i 

storage where the tactical talues for sequences of all lengths | 

i 

are found# See Table XI for formula® to calculate the address# i 

i 

Since we wish the computer to learn to play the game 9 j 
Its sp ee d will be greatly affected by the slowness of Its i 

I 

in-put-output as cospared with its cosputation speed# The 
delay created by a human opponent* e thought is relatively 
eBoraosase 4% m# iaiqnhewm «ws» 10 m$m mt 

leaning progr amme play against an * Intelligent Programme*} 
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Ho* of digits 
in a seiuenee 

4rea of storage vimre the 
tactical rallies are found 

1 

1 to 7 

8 

8 to 40 

5 

so to tm 

4 

860 to 1099 

8 

1100 to 8619 

6 

5680 to 8689 

7 

3660 to 18700 


The «wnaeuc®» are stored in the following order, starting 
location 1, ending location 15700 $ 


1 

35 

71 

♦ 

* 

8 

36 

* 


# 

8 

57 

♦" 

7684 

♦ 

• 

41 

76 

18548 

* 

* 

48 

185 

m 

1S84&07 

7 

45 

* 

♦ 

♦ 

18 

48 

* 

# 

•# 

15 

45 

187 

• 

* 

14 

47 

158 

* 


18 

81 

* 

# 

* 

16 

68 

* 

* 

* 

17 

85 

• 

# 

# 

81 

84 

213 


t 

85 

fl£J& 

w? 

* 

76545 

* 

84 

57 

• 

« 

♦ 

28 

61 

a 

• 

* 

26 

68 

«" 

m 

♦ 

gift MtM 
2? 

65 

766 

m 

# 

51 

64 

1854 

# 

# 

58 

WJ> 

i Hif 

68 

67 

• 

* 

* 

«r 

7684581 
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Ha# Formulae to calculate to® addresses *t*ere tUe taotle&S 
values various sequences are stored Ct tm addrpfiaes are Identified 
tey the avateoi IE)* 

1* for sequences with one digit: 2a*XDX (see Site}* 

£* For sequences with two digits! 

m • ? ♦ IPS, ^uere 

B>£ « «{m*l) ♦ IB £* 

3# For sequences with three digit®: 

m»7*7*6*IP& t 
IP3-5 (IP2*1) +ID5 • 

4* For sequences with four digits: 

IE *7 +7*6*7*6#5»214 » 

I* For sequences with fly® digit©: 

Ift *7*7 *6*7 *6 *6*7 *6*6*4-*IP5 9 
IP8*3(m-l)*lD6 * 

6* for sequences with sis digits: 

M*7^7^7»6^7*6,6«4^7*i^#4^m t | 

i?6-s(iP5^i)» m * 

7* For sequences with seven digit®: | 

H*7«-7 •«^.4.^7*4*fi#^7»i*i#4#W f *i#S,4*»#t*Ii| 

m*im*i>+wr « I 

Site* IDI, IDS etc* are found tins: The seven digits are stored Ini 

a register as *frM»wra ' 

J ItS 4 6 6 7 

2T4B{J) 1 fi 3 4 3 6 7 
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TABI® II (CONED*) 

Tills Is referred to as the TAB register In the actual 
program©. 3 Is an Index and this a ©sums the sane Yalue 
as the actual number chosen by either program©* Once a 
digit Is selected as a mote. It Is * removed* frc© the register 
and all the digits to the rlefct of this digit are reduced by 1. 
For example. If 5 Is the number selected first, ID1*3* The 
altered register would now look like this! 

3 1234667 

ITAB(J) 1 2 3 3 4 5 6 

Instead of actually removing 3 by making It zero or by 
other means me hare left it as it ms In the register earlier* 
Since no digit is selected twelce In the same game, this does 
not matter. If now, 5 in the next choice, then, IB2*JT4B(5)*4* 
The register will now appear thus! 

3 1234567 

IT&SCjr) 1 2 3 3 3 4 5 

This is repeated ererytloe a digit Is chosen* 


I 
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with the earlier moves of the game has a positive or negative 
tactical vale# associated with it# If It is positive t that 
choice Is accepted# If negative # it will choose the next 
Ware and repeat the process# It will not sales moves which 
have led to a loss In earlier games, after it has played a 
number of games# Its moves will be so as to lead the gams 
to a ^aw# V© then say that the learning Programme has 
•learnt* • 

One other Important aspect needs to be mentioned# 

Any game can be regarded as a tree, the nodes representing 
the *m oves* and the branches, the •choice* * for tietaotoe, 
a tree would look like figure 4#2# T fe ls does not represent 
any particular game of coarse, bat only displays the several 
choices that become apparent at each move# After ware 9 
has been chosen, ISEmo has a choice of 7 wave# for each 
ware chosen by iNrsao, waiso has six choices* And so on# 
How, look at figure 4#3# This is the complete tree for a game 
where INTPSO has chosen ware 1 as Its first choice# I3S4PBQ 
now has a choice of $ wares and a choice of any of these 
smarts except ware i will lead to a loss for aim®# 
According to its scheme, the if Amo will first play ware 2, 
lose the game and sake the tactical value for the sequence 
Cl, 2) | negative# A similar fate will overtake sequences 
(1,2) and (1,4)# square 2 has now been discovered to be the 
safe choice# Suppose IKHO now plays ware 2# J»*mo again 
has a choice of four squares and it will loam to play ware 
6 only after losing two more games by playing squares I and 4# 
is a result sequences (i ,2,2,2) and (i, 2,2,4) have negative 




tactical values* The lUTpao may next play square 5* If 
EBIlESO plays square 4, as it does the first time, it loses* 
Sequence (1,5,2, 6,3, 4) now has a negative tactical value* 

Square 7 is the safe choice and the game now ends in a 
draw. So far, altering of tactical Values of •had* sequences 
has not presented a problem. 4s soon as INTPEtO wins a game, 
the Executor calls the ISOSG and IE4PE0 immediately changes 
the tactical value for the sequence ending in its move* 

But supposing a situation as shown in figure 4.4a 
occurs* Such a position is called a "fork" and according to 
Its scheme IS4EB0 would play squares 1, 3, 5, 6, and 7 and 
lose all the timet 4pparently for this game, any correction 
at this stage is of no avail, so one has to effect rectification 
at an earlier move* ipparently square 1 itself was a had 
choice, so the sequence ( 4 , should have a negative tactical 
value* The EB4Eao should he able to detect such cases, go 
hack and alter a previous choice* 4 tree for this game is 
shown in figure 4*4h* No satisfactory choice exists at node B* 
One has to go hack to node 4 and make a different choice* For 
this reason, the EB45RQ is provided with two extra registers, 
where the addresses of previous sequences can he stored* 

These are identified as IB i and m2, in the programme* Thus 
at any stage of the gams, we have available to us the addresses 
of three different sequences, one corresponding to the latest 
move (in register Id) , and two, corresponding lew to two 
previous moves (In registers Hi and 132) • Once a fork has 
Been detested, the IB41E& will make the tactical value at the 
address stored In XB1 negative and instead of continuing the 








4- Fogtc' u r^voR or LE.APRQ 
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ga»» •will concede it* 

Flov Charts V, TI **nd YII show the subprogrammes 
AhfflS, TUB and iMST-which supplies i®mo with the least 
vacant siaare* These dre sufficiently clean and need no 
explanation* 

4*5 Programing detail# and results 5- 

(a) The programs ms sun on an IBM 7044 computer* 

The programing language used ms FORTRAN 3Y t except for the 
RANDOM subprogramme ifclch ms coded in MAP, the internal 
langoa<*e of the Machine* 4 aeries of ten-thoasand games were 
played “between the INTP90 and the ISAPR0* Running time ms 
about seven minutes including compilation time* The approximate 
else of the programs ms! 

1* Storage spaee for tactical values 15700 locatlot 

2* Storage space for the hoard* 

■■ 

for storing the sons and for | 

the TAB register 24 locatloi; 

i 

5* Total number of Instructions 214 inatruc* 

tlons ; 

(h) Firstly* the DJTPao ms made to play against a 
random player, with no learning incorporated* since the retained 
random generator ms already available, the ixecutor simply 
called the Bn™) and RANDOM sahprogrammes alternately* It 
ms discovered that the random player drew,on an average about 
flve-per cent of the games it played* This formed a basis for 
comparison with the performance of the learning Programme* 

The performance curve for the game between umao and the random 
player le shorn In figure 4*5a* 



XERCEMT OF 0RAVKJ GAMES 



4 . 5 g« j .. 


g^Lfw y^g Imtp^ Rak,^ 

AMfi Intpto vs. Lea PRC, 


^*7£nl 


(©) Ifeart* the lEOBBS played a series of games 
with IKfB&Q* It drew its first game after posing the 
first twenty gases* it the end of fifty games, it had 

dram id and lost the rest* it the end of [a hundred games, 

/* 

It had lost only 41 and dram the rest • add encouraging 
improvement in its performance* ifter thatj IMD coaid 
win omy nine more games, thas making a total of fifty 
wins, after which it won no more games* Thus, stodgy 
state ms reached after fifty games* Figure 4.8 shows 
the "learning carve" fear this series of contests* 1 few 
words aboat learning curses ere in order here* Since 
learning is an inference from performance, m mast haw© 
some index for the measurement of performance* Learning 
curves ara usually plotted with this index as one coordinate 
and time or number of attempts or any such similar variable 
as the otter coordinate* In oar case, the coordinates were, 
the games that were drawn, expressed as a percentage of the 
total number of games played, and the total number of games 
that were played* Learning carves are omy dftta obtained 
in alleged learning situations arranged so as to thickly 
reveal a change in behaviour or performance* the manner 
in which the rise occurs is an important feature* the rise 
might occur with less and less improvement between trials 
or with mom and more improvement* the stajflry sharply 
rising curves, according to psychologists, ate supposed 
to reflect "insight" or some kind of sudden solution of 
a problem, the realisation of a significance, the perceptual 


reorganisation of a field and so on (C-uthrie, 1935)* 

The car*© shorn in figure 4*5 shows a rradoal rise which 
is typical of trial~and-error learning situations or learning 
from past experiences# 

(d) in interesting situation ^tiich occurred in 
the course of these runs deserves mention* The following 
series of metres appeared* 7-3»6~2«l*4, where the first 
choice is by IUMO. Clearly, this is a fork and in favour 
of ESOTOI XBftlEO actually won this particular game and 
it was also discovered that this ms the only game it ever 
won against IHTKtO* such situations are not many 5 this 
mm on© was purely accidental* See figure 4*6 for the 
•board state* of this game* 

4*4 % have shown that the scheme of remembering 

sequences of moves and associating good (♦we) and bad 
C-ve) tactical values with them works for the game of 
tictactoe* Though only a limited number of games were 
played, it is clear that this scheme would work against 
any tactic by the opponent* The first tee moves were 
restricted in order to reduce the demands on memory space 
remind* 4 variation on this could be effected If 
changing the numbering of the board* For example, we could 
have the arrangement of figure 4*7a* In this ease, we would 
only have to alter the definition of the row, column and 
diagonal sums* &nd, XE4FS0 will continue to draw all the 
games* Or, we can have the alternates of figures 4*7b and 
4*?c§ and things would be the sane* *s can then say that 
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t he Learning Programs has dereloped concepts of mamsttf 
that are inherent in this gaise* Figure 4*74 shows the 
hoard used in the present case for immediate comparison 
with the other three figures* 
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T1 m fact that computers hair© heen aaecessfaiiy 
programmed not only to play games like tletaotoe, checkers 
and chess hot also improve their performance ©1th play* 
experience leads on© to he Here that* lea rning* or learning* 
like hehayicnr can he displayed hy man*aade systems* This 
Is a significant step towards realisation of "artificial 
intelligence* - systems performing tasks, which if performed 
hy man would he regarded as revering intelligence# There 
exists already, computer programmes which prose mathematical 
theorems, pattern recognition programmes, q*ie at ion*answ©r ing 
machines, programmes simolating concept formation in humans, 
social behaviour etc# 

Since oar main interest in this thesis has been 
in gam»«playlng programmes, it seems a fitting end to <joote 
teannon’s words regarding mm playing machines and. their 
human opponents: *0n an equal time ha sis, the speed, patience 
and deadly accuracy of the machine would he telling against 
human fallibility** 
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APPENDIX 



Place instructions ccntainins bcth rules and 

TACTICS IN KEISTER A. 


Store TACTICAL VALUES IN PECISTE R B- THRSE 
"REPRESENT ' VALUES* OF MOVES MADE BY . 


pLArCfe INSTRUCTIONS CONTAINI 
POLES ONLY IN B . 


V/D+EN BOARD STATE 15 PRESENTED Tfi A , IT FILLS IN A 
VACANT StpOARE ACCORCSM^ TC ITS EXPLICIT IKSTROCTlCNS 
It DITEN TRANSFERS RECORD Cf BOARD STATE 

TO B. 


VlHEN '&CWRD STATE 15 PRESENTED TO & v |T FlBST CHEOS 
IF TD+R PREVIOUS MOVE ©y'A' VO As A Y/lMNlM^ MOYE- IF 
50, IT MEANS IT HAS LOST THE CAME AND 'T WILL 

"Promptly mare the tactical valoe for the se-Coence 
OF ITS MOVES IN THIS CAME > NE6ATI YE • 

It t hen Proceeds to a fresh came., re torn ws 
Empty -board state to A. If the came is not 

Lost, it -PROCEEDS to NEXT 5T&P. 


If the CAME 15 Nor OVER (PRAW OP WIN FOR B ) 
B SELECTS A SQUARE WITH POSITIVE TACTICAL 
VALUE AND PLAYS IT- It 3)C£S N-Df T>LAY S^uA-RES 
V/\TH -VE TAC- VALUES , ll THEN TRANSFERS BOARD 

State to A . 


-LOW Lo+abt 


AMR Between) two tro6Rams in the 


Same compotes . 
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Stak 


Set COUNTERS TC ZERO: lNT>*TNj =Oj LEAWIN-O j Tl E - O . 


Start 

<£*AN4E 

± n z z _ 

All NINE SQUARES 3+AVE WEI6WT ZERO. 

1 $ COLUMN t DIAGONAL <(JM5 A>-R£ ^EGO- Mo^£ 
C.OUNTER NN=C. £tcRE Z:E!?as IN R£<5 IST£rs XRl ^rR2. 


A US MEN! MOVE 
COUNTER BV d. 
NNU NM + 4. 


IS MM OOD/igyt 


EVEN 


Call Imtcu-jc 

TteDERAM 


call Learns 
Trcs ram 


I^IVE WEI<5AT I 
d To SELECTED 
5LUARE 


Call 'auras' 


Call' tab' 
Subprogram 


Compute row, colvt Diagonal sums 


G > ive -weto+T 

A TV SELECTED 
SGJUARE 



AUGMENT 
TIE COUNTER 
BY A- 


. e came ? 


AUGMENT \v 

IMTWIN COLNS~.- a--— 
12>y ONE 


-<''700 ES-^ 

INTPRO WIN 


augment 

^EAPRD *" LEAWIN C0UNT6 

©y dl 


CALL LEAPRO 
FOR TAC yAL. 
O+AUQe 


Flow Chart II : ' Executor 
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Pl_A Y VACANT 
THIRD SQUARE 


SELECT RANDOM SQUARE 
(cALi- 'RANDOM' SUfePKCS^A^i 



"ElAy selected Skuaer 


(Return) 

Plow C>\ art III : 'Imtprg' 5 obprogram. 



Plow (Saprt IV : Leaped' Subprogram 










Enjte r 


IT - — . 

First entry cf 
''~^-45rst 


SET Tactical Values 
For all Sequences 
£6dal rc> unity 




C>ieac Move Number 

. — .1 - H Z 

Calculate adeess 

OF SEQUENCE FCRNfCt 
Us INC FORMULA 


Fl6W . C34ART V : 


^ETORNL 


AtORE S 


^BPROSRAM 


Is it first 

: NTRy Cf (^AMJ 


Yes Store digits 4 
1 *“ To ~7 IN ITAB 

Reciter 


-""'There digits^ 

uro RiciircF^nr, 

\£MCSEM 


Subtract 1 from 
All digits -to rig>t 

©f= DIGIT CHiSEM 


,T$E TORI' 


'Uo w C >. 


art 


tab 


AuB'PROftEAM 
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